Data-intensive Bioinformatics Research Articles

Improving Performance of Hardware Accelerators by Optimizing Data Movement: A Bioinformatics Case Study

Modern hardware accelerator cards create an accessible platform for developers to reduce execution times for computationally expensive algorithms. The most widely used systems, however, have dedicated memory spaces, resulting in the processor having to transfer data to the accelerator-card memory space before the computation can be executed. Currently, the performance increase from using an accelerator card for data-intensive algorithms is limited by the data movement. To this end, this work aims to reduce the effect of data movement and improve overall performance by systematically caching data on the accelerator card. We designed a software-controlled split cache where data are cached on the accelerator and assessed its efficacy using a data-intensive Bioinformatics application that infers the evolutionary history of a set of organisms by constructing phylogenetic trees. Our results revealed that software-controlled data caching on a datacenter-grade FPGA accelerator card reduced the overhead of data movement by 90%. This resulted in a reduction of the total execution time between 32% and 40% for the entire application when phylogenetic trees of various sizes were constructed.

Linking big biomedical datasets to modular analysis with Portable Encapsulated Projects.

BackgroundOrganizing and annotating biological sample data is critical in data-intensive bioinformatics. Unfortunately, metadata formats from a data provider are often incompatible with requirements of a processing tool. There is no broadly accepted standard to organize metadata across biological projects and bioinformatics tools, restricting the portability and reusability of both annotated datasets and analysis software.ResultsTo address this, we present the Portable Encapsulated Project (PEP) specification, a formal specification for biological sample metadata structure. The PEP specification accommodates typical features of data-intensive bioinformatics projects with many biological samples. In addition to standardization, the PEP specification provides descriptors and modifiers for project-level and sample-level metadata, which improve portability across both computing environments and data processing tools. PEPs include a schema validator framework, allowing formal definition of required metadata attributes for data analysis broadly. We have implemented packages for reading PEPs in both Python and R to provide a language-agnostic interface for organizing project metadata.ConclusionsThe PEP specification is an important step toward unifying data annotation and processing tools in data-intensive biological research projects. Links to tools and documentation are available at http://pep.databio.org/.

Bringing up the bio-datafied child: scientific and ethical controversies over computational biology in education

ABSTRACT Scientific advances in genetic analysis have been made possible in recent years by technical developments in computational biology, or bioinformatics. Bioinformatics has opened up the human genome to diverse analyses involving automated laboratory hardware and machine learning algorithms and software. As part of an emerging field of social genomics, recent educational genetics studies using big data have begun to raise challenging findings linking DNA to predicted life outcomes. Bioinformatic technologies and techniques including ‘genome-wide association’ and ‘polygenic scoring’ are producing new kinds of genetic biodata and expert knowledge for rethinking the upbringing and education of children. This article takes as an empirical opening a recent epistemic and ethical controversy over the use of biodata to make genetic predictions about educational, socio-economic and life outcomes, detailing the forms of expert knowledge and technologies involved these predictions, and exploring the social and ethical implications of data-intensive bioinformatics.

Using simple PID-inspired controllers for online resilient resource management of distributed scientific workflows

Scientific workflows have become mainstream for conducting large-scale scientific research. As a result, many workflow applications and Workflow Management Systems (WMSs) have been developed as part of the cyberinfrastructure to allow scientists to execute their applications seamlessly on a range of distributed platforms. Although the scientific community has addressed this challenge from both theoretical and practical approaches, failure prediction, detection, and recovery still raise many research questions. In this paper, we propose an approach inspired by the control theory developed as part of autonomic computing to predict failures before they happen, and mitigated them when possible. The proposed approach is inspired on the proportional–integral–derivative controller (PID controller) control loop mechanism, which is widely used in industrial control systems, where the controller will react to adjust its output to mitigate faults. PID controllers aim to detect the possibility of a non-steady state far enough in advance so that an action can be performed to prevent it from happening. To demonstrate the feasibility of the approach, we tackle two common execution faults of large scale data-intensive workflows—data storage overload and memory overflow. We developed a simulator, which implements and evaluates simple standalone PID-inspired controllers to autonomously manage data and memory usage of a data-intensive bioinformatics workflow that consumes/produces over 4.4 TB of data, and requires over 24 TB of memory to run all tasks concurrently. Experimental results obtained via simulation indicate that workflow executions may significantly benefit from the controller-inspired approach, in particular under online and unknown conditions. Simulation results show that nearly-optimal executions (slowdown of 1.01) can be attained when using our proposed method, and faults are detected and mitigated far in advance of their occurrence.

Large-scale parallel genome assembler over cloud computing environment.

The size of high throughput DNA sequencing data has already reached the terabyte scale. To manage this huge volume of data, many downstream sequencing applications started using locality-based computing over different cloud infrastructures to take advantage of elastic (pay as you go) resources at a lower cost. However, the locality-based programming model (e.g. MapReduce) is relatively new. Consequently, developing scalable data-intensive bioinformatics applications using this model and understanding the hardware environment that these applications require for good performance, both require further research. In this paper, we present a de Bruijn graph oriented Parallel Giraph-based Genome Assembler (GiGA), as well as the hardware platform required for its optimal performance. GiGA uses the power of Hadoop (MapReduce) and Giraph (large-scale graph analysis) to achieve high scalability over hundreds of compute nodes by collocating the computation and data. GiGA achieves significantly higher scalability with competitive assembly quality compared to contemporary parallel assemblers (e.g. ABySS and Contrail) over traditional HPC cluster. Moreover, we show that the performance of GiGA is significantly improved by using an SSD-based private cloud infrastructure over traditional HPC cluster. We observe that the performance of GiGA on 256 cores of this SSD-based cloud infrastructure closely matches that of 512 cores of traditional HPC cluster.

Experiences with workflows for automating data-intensive bioinformatics.

High-throughput technologies, such as next-generation sequencing, have turned molecular biology into a data-intensive discipline, requiring bioinformaticians to use high-performance computing resources and carry out data management and analysis tasks on large scale. Workflow systems can be useful to simplify construction of analysis pipelines that automate tasks, support reproducibility and provide measures for fault-tolerance. However, workflow systems can incur significant development and administration overhead so bioinformatics pipelines are often still built without them. We present the experiences with workflows and workflow systems within the bioinformatics community participating in a series of hackathons and workshops of the EU COST action SeqAhead. The organizations are working on similar problems, but we have addressed them with different strategies and solutions. This fragmentation of efforts is inefficient and leads to redundant and incompatible solutions. Based on our experiences we define a set of recommendations for future systems to enable efficient yet simple bioinformatics workflow construction and execution.ReviewersThis article was reviewed by Dr Andrew Clark.

Introduction to the Special Issue on Cloud for Health

It is a pleasure to present this special issue covering subjects related to applying cloud computing to bioinformatics, biomedicine, and health, including solutions to area problems and architectural adaptation of cloud systems to fit those problems. The special issue includes papers selected from the workshops C4Bio 2014 and CCGrid-Health 2014 that, after further extension and additional review, were selected for publication. C4Bio 2014 and CCGrid-Health 2014 were held in Chicago, during May 26-29th 2014 within the framework of the 14th IEEE/ACM International Symposium on Cluster, and Grid Computing (CCGrid 2014). The special issue begins with the review paper Cloud Computing in Healthcare and Biomedicine. Calabrese and Cannataro explore the current academic and commercial cloud-based solutions developed in the healthcare and biomedicine domains, designed in order to satisfy the specific needs (e.g. massive scalable computing and storage, data sharing, on-demand access anytime and anywhere) in these fields. Subsequently, the paper summarizes main issues regarding the use of Computing in healthcare and biomedicine. The article Selected Approaches and Frameworks to Carry out Genomic Data Analysis on the Cloud, by Church and Goscinski, examines commonly used cloud-based genomic analysis services, introduces the approach of exposing data as services and proposes two new solutions, namely HPCaaS and Uncinus, which aim to automate service development, deployment process and data provision. By comparing and contrasting these solutions, the authors identify key mechanisms of service creation, execution and data access required to support non-computing specialists employing clouds. Ahmad, Hasan, and Siddique propose to formally reason about molecular pathways of Zsyntax based deduction using the HOL4 theorem prover and developing a biologist friendly graphical user interface. On the Formalization of Zsyntax with Applications in Molecular Biology shows additionally a use case with the formal analysis of cancer-related molecular pathway, i.e., TP53 degradation and metabolic pathway, known as Glycolysis. The article Accelerating Comparative Genomics Workflows in a Distributed Environment with Optimized Data Partiotioning and Workflow Fusion, by Choudhury et al., discusses different strategies for alleviating the computational constraints experienced by several data-intensive bioinformatics applications, presenting a workflow-based framework to parallelize the compute-intensive tasks of genome alignment and variant calling, the two most important stages in most comparative genomics applications. This framework provides workflow fusion to merge multiple sequential workflows into a single optimized workflow that benefits from caching, eliminating choke points, and stacked partitioning. In Performance Comparison and Tuning of Virtual Machines For Sequence Alignment Software, Estrada et al. explore the performance cost of virtualization for the fast growing application domain of genomics to inform the feasibility of running an NGS pipeline in a cloud, and in doing so consider two prevalent short-read sequence alignment programs, BWA and Novoalign. They execute these applications in three separate open-source system virtualization solutions (KVM hypervisor, Xen paravirtualized hypervisor, and Linux Containers) and compare the runtime in each environment against the runtime of the same system without virtualization and measure the relative performance of each hypervisor, presenting tuning suggestions for cloud implementors and users. The article Extending XNAT towards a Cloud-based Quality Assessment Platform for Retinal Optical Coherence Tomographies, by Jansen et al., presents a platform designed for automatic quality assessment of retinal OCTs and provided as a cloud-based service employing OpenStack. It extends the image management platform XNAT by services to calculate and store quality measures. It is also extensible regarding new quality measure algorithms, allowing the developer to upload, compile and test code for the system's architecture. Finally, the paper A Tool for Managing the X1.V1 Platform on the Cloud, by Marzini et al., proposes a framework for managing the execution of the X1.V1 platform on the Cloud. This framework enables an easy, quick, and secure management of the resources allocation and reallocation to X1.V1 Virtual Machines, in order to enhance the platform performance, optimize resource utilization and, consequently, reduce the whole services cost. We would like to thank all the reviewers for their effort, cooperation, and valuable feedback and all the authors who submitted papers to C4Bio 2014 and CCGrid-Health 2014 and to this Special Issue.

BioPig: a Hadoop-based analytic toolkit for large-scale sequence data

The recent revolution in sequencing technologies has led to an exponential growth of sequence data. As a result, most of the current bioinformatics tools become obsolete as they fail to scale with data. To tackle this 'data deluge', here we introduce the BioPig sequence analysis toolkit as one of the solutions that scale to data and computation. We built BioPig on the Apache's Hadoop MapReduce system and the Pig data flow language. Compared with traditional serial and MPI-based algorithms, BioPig has three major advantages: first, BioPig's programmability greatly reduces development time for parallel bioinformatics applications; second, testing BioPig with up to 500 Gb sequences demonstrates that it scales automatically with size of data; and finally, BioPig can be ported without modification on many Hadoop infrastructures, as tested with Magellan system at National Energy Research Scientific Computing Center and the Amazon Elastic Compute Cloud. In summary, BioPig represents a novel program framework with the potential to greatly accelerate data-intensive bioinformatics analysis.

PoPLAR: Portal for Petascale Lifescience Applications and Research

BackgroundWe are focusing specifically on fast data analysis and retrieval in bioinformatics that will have a direct impact on the quality of human health and the environment. The exponential growth of data generated in biology research, from small atoms to big ecosystems, necessitates an increasingly large computational component to perform analyses. Novel DNA sequencing technologies and complementary high-throughput approaches--such as proteomics, genomics, metabolomics, and meta-genomics--drive data-intensive bioinformatics. While individual research centers or universities could once provide for these applications, this is no longer the case. Today, only specialized national centers can deliver the level of computing resources required to meet the challenges posed by rapid data growth and the resulting computational demand. Consequently, we are developing massively parallel applications to analyze the growing flood of biological data and contribute to the rapid discovery of novel knowledge.MethodsThe efforts of previous National Science Foundation (NSF) projects provided for the generation of parallel modules for widely used bioinformatics applications on the Kraken supercomputer. We have profiled and optimized the code of some of the scientific community's most widely used desktop and small-cluster-based applications, including BLAST from the National Center for Biotechnology Information (NCBI), HMMER, and MUSCLE; scaled them to tens of thousands of cores on high-performance computing (HPC) architectures; made them robust and portable to next-generation architectures; and incorporated these parallel applications in science gateways with a web-based portal.ResultsThis paper will discuss the various developmental stages, challenges, and solutions involved in taking bioinformatics applications from the desktop to petascale with a front-end portal for very-large-scale data analysis in the life sciences.ConclusionsThis research will help to bridge the gap between the rate of data generation and the speed at which scientists can study this data. The ability to rapidly analyze data at such a large scale is having a significant, direct impact on science achieved by collaborators who are currently using these tools on supercomputers.

A Survey of Bioinformatics Applications on Parallel Architectures

Based on bioinformatics algorithm, there are a wide range of implementations. With the urge for program speed, many applications take the heuristic approach to compensate running time. One of the most critical shortcomings of this technique is the loss of optimality, i.e. the desired results may not always be found. To overcome this problem, many different hardware architectures have been experimented for bioinformatics algorithm such as cell broadband engine, cluster and compute unified device architecture where, the main technique for obtaining high performance is to parallelize the task to be run simultaneously by multiple vector execution units with single instruction multiple data and by multiple processors with multiple instruction multiple data. In this paper we presents a survey of data intensive bioinformatics applications on variety of parallel Architecture that are available for accelerating the processing of large biological data set .

The use of grid computing to drive data-intensive genetic research

In genetics, with increasing data sizes and more advanced algorithms for mining complex data, a point is reached where increased computational capacity or alternative solutions becomes unavoidable. Most contemporary methods for linkage analysis are based on the Lander-Green hidden Markov model (HMM), which scales exponentially with the number of pedigree members. In whole genome linkage analysis, genotype simulations become prohibitively time consuming to perform on single computers. We have developed "Grid-Allegro", a Grid aware implementation of the Allegro software, by which several thousands of genotype simulations can be performed in parallel in short time. With temporary installations of the Allegro executable and datasets on remote nodes at submission, the need of predefined Grid run-time environments is circumvented. We evaluated the performance, efficiency and scalability of this implementation in a genome scan on Swedish multiplex Alzheimer's disease families. We demonstrate that "Grid-Allegro" allows for the full exploitation of the features available in Allegro for genome-wide linkage. The implementation of existing bioinformatics applications on Grids (Distributed Computing) represent a cost-effective alternative for addressing highly resource-demanding and data-intensive bioinformatics task, compared to acquiring and setting up clusters of computational hardware in house (Parallel Computing), a resource not available to most geneticists today.

Editorial: Special Section on High-Performance Computational Biology

OVER the past decade, computational molecular biology has grown into a mature discipline with a well-defined body of core knowledge, and participation from a large and diverse group of researchers. To keep pace with the explosive growth in research in this field, a number of high quality journals and annual conferences have been established. Many universities are actively building academic programs and research centers and groups in computational biology. As a reflection of the maturing of the field, numerous textbooks on computational biology and its various subtopics have been written in recent years, and undergraduate programs are underway. Despite this progress, computational biology continues to be a vibrant discipline with many outstanding research problems and potential for new avenues of investigation for decades to come. We broadly view high-performance computational biology as the development and application of high-performance computing techniques for extending the reach or scale of investigations in computational biology. A major component of this is the development of parallel and distributed algorithms, and programming environments and systems for aiding biological investigations using highperformance parallel computers, grid computing, and emerging architectures. There is a compelling need for such research given the explosive growth in biological information, the complexity of interactions that underlie many biological processes, and the diversity and interconnectedness of organisms at the molecular level. However, research in high-performance computational biology has not grown as rapidly as computational biology itself. There are subfields of computational biology which have not seen significant influx of ideas from the high-performance computing community. This is perhaps a reflection of the confluence of expertise needed to conduct research in high-performance computational biology, which sets up a barrier to entry for new researchers. Efforts spent in transgressing the barrier are worthwhile given the opportunities for high impact research. By bringing together research in this area as a special section, we hope to provide a resource for IEEE Transactions on Parallel and Distributed Systems (TPDS) readers interested in this field and aid the entry of new researchers into the field. The arguments in favor of a sustained effort in highperformance computational biology are stronger than ever. New high-throughput sequencing machines introduced within the last year, such as those from 454 Life Sciences Inc., have significantly accelerated sequencing capabilities. Using 454 sequencing systems, it is possible to sequence as many as 200,000 short DNA fragments in a 4 hour experiment for a few thousand dollars. These machines are increasingly being used to sample transcriptomes of many organisms. The sequencing of several complex plant genomes is underway starting with maize and sorghum. Similar to large-scale genome sequencing projects, comprehensive gene expression profile measurement projects are underway to conduct large-scale microarray experiments on an organism spanning various organs, diesease/stress induced states, and developmental stages. Forays into personalized medicine, rational drug design, large-scale systems biology, such as the study of protein-protein interaction networks at the whole organism level, understanding evolutionary relationships and building the tree of life, all require processing vast amounts of data or carrying out highly complex computational tasks. In this special section, we showcase some of the recent work in high-performance computational biology. In addition to the open call for papers, authors whose work was published in the 2005 IEEE International Workshop on HighPerformance Computational Biology (HiCOMB, http:// www.hicomb.org) were solicited to submit extended versions of their papers. Each manuscript submitted to the special section was subjected to rigorous, independent peer review by three to four reviewers. We are extremely grateful to all the reviewers who agreed and delivered on providing thoughtful reviews within the time constraints imposed for the special issue. Based on the reviewer suggestions and our own reading of the manuscripts, six manuscripts were selected for publication in the special section. The first paper in this special issue is on a scalable implementation of the widely used BLAST search program for homology detection between a query sequence and a database of known sequences. In “ScalaBLAST: A Scalable Implementation of BLAST for High-Performance DataIntensive Bioinformatics Analysis,” Christopher Oehmen and Jarek Nieplocha report on ScalaBLAST, a high-performance sequence alignment program they developed to enable applications that require thousands to millions of queries to be performed simultaneously. Such queries are used in applications such as multiple genome/proteome comparisons, and in finding genes in newly sequenced genomes. By using a combination of techniques, including target database distribution, exploiting multilevel parallelism, parallel I/Os and latency hiding, the authors achieve a scalable implementation of this ubiquitous search program. IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, VOL. 17, NO. 8, AUGUST 2006 737

Visualizing Biological Pathways: Requirements Analysis, Systems Evaluation and Research Agenda

Pathway diagrams are used by life scientists to represent complex interactions at the molecular level in living cells. The recent shift towards data-intensive bioinformatics and systems-level science has created a strong need for advanced pathway visualizations that support exploratory analysis. This paper presents a comprehensive list of requirements for pathway visualization systems, based on interviews conducted to understand life scientists' needs for pathway analysis. A variety of existing pathway visualization systems are examined, to analyze common approaches by which the contemporary systems address these requirements. A heuristic evaluation, by biology domain experts, of five popular pathway visualization systems is conducted to analyze the end-user perception of these systems. Based on these studies, a research agenda is presented concerning five critical requirements for pathway visualization systems. If addressed effectively, these requirements can prove to be most helpful in supporting exploratory pathway analysis. These include: (1) automated construction and updating of pathways by searching literature databases, (2) overlaying information on pathways in a biologically relevant format, (3) linking pathways to multi-dimensional data from high-throughput experiments such as microarrays, (4) overviewing multiple pathways simultaneously with interconnections between them, (5) scaling pathways to higher levels of abstraction to analyze effects of complex molecular interactions at higher levels of biological organization.